A Re-examination of the Community of Inquiry Framework: Social Network and Content Analysis

This study provides a simultaneous examination of all components of the Community of Inquiry (CoI) framework (Garrison, Anderson, and Archer, 2000; Anderson, Rourke, Garrison, & Archer 2001; and Rourke, Garrison, Anderson & Archer 1999) and seeks to extend previous work into the nature, development, and relationships between the constructs of “presence” hypothesized in the model. To accomplish this goal we use content and social network analysis to examine the discourse produced among all participants in two semester-length online asynchronous college courses. Coding for the existence and relative intensity of forms of presence we identify patterns and relationships between instructors’ and students’ teaching presence and social presence and the corresponding quantity and levels of cognitive presence that emerges. The study reveals complex relationships between these variables that have implications for the development of higher order thinking and meaningful learning in online environments. Study findings also have implications for online teaching practice and ongoing research on the CoI framework. Introduction and Objectives Garrison and Arbaugh (2007) make a convincing case that the Community of Inquiry (CoI) framework has become one of, if not the leading models guiding research into online teaching and learning in higher education. Observing that the original article that laid out the framework has been cited in more than 225 other published articles, the authors note that this is the most frequently cited paper in the Elsevier journal, The Internet and Higher Education. The other foundational articles for the CoI model have had an equally impressive impact on research in this area. In all the initial journal articles by Garrison, Anderson, and Archer (2000), Anderson, Rourke, Garrison, & Archer (2001), and Rourke, Garrison, Anderson & Archer (1999) have been cited by nearly 1000 other papers in the field of online learning research. The number of citations continues to rise and a relatively vigorous research community focusing on CoI has begun to emerge (e.g. Swan, Richardson, Ice, Garrison, Cleveland-Innes, & Arbaugh, 2008). In this paper we examine the CoI framework and raise conceptual and methodological issues that we feel characterize previous research. It is our goal to advance research in online learning and the development of the CoI model by approaching the enterprise with new conceptual and methodological tools. Theoretical Framework The CoI framework posits that knowledge construction in learning environments occurs through the development of a community of inquiry (Pierce, 1955; Lipman, 2003) characterized in online education specifically by optimal levels of teaching, social, and cognitive presence. (Garrison, Anderson, & Archer, 2000). The model assumes that, in the absence of face-to-face interaction, participants in online learning environments must strive to recreate the social and knowledge building processes that occur via moment by moment negotiation of meaning found in the classroom. These dynamics are depicted by the concepts of presence described in more detail below. Reflecting recent understanding of collaborative pedagogy, teaching presence refers to the instructional design and organization, facilitation of productive discourse, and direct instruction developed in online courses, ideally by both instructors and students (Anderson et. al 2001). The social presence construct in the model builds upon past research (e.g. Short, Williams, & Christie, 1976) attempting to understand how participants in mediated communication project themselves as “real people”, especially in the relatively lean medium of fully text-based, asynchronous communication. Also reflecting the notion of teacher immediacy (Mehrabian, 1966) the model hypothesizes modes of social presence including textual demonstration of affect, group cohesion, and open communication necessary to establish a sense of trust and, ideally, membership in a community dedicated to joint knowledge construction. It should be noted that the notion of teaching presence and social presence combine with each other and the third construct, cognitive presence to develop such a community. In the CoI model, cognitive presence is seen as developing through a series of four cyclical stages beginning with a triggering event then moving (again ideally) to exploration, integration and resolution. The concept of cognitive presence is thus built upon the Dewian notion of practical inquiry (Dewey, 1933, 1959) and reflects both critical and creative thinking processes. Research on the CoI framework documents the correlation between components of the model and learner satisfaction and perceived learning (e.g. Shea et al 2004, Swan & Shih, 2001) and to online learner sense of community (Shea et al, 2006). Additional studies have added to our understanding of the coherence of the theoretical component as a research tool (Arbaugh, 2007; Arbaugh et al 2008; Shea et al 2006; Shea & Bidjerano in press). Lacking in this research is an attempt to test hypotheses generated by previous work documenting online knowledge construction through the interaction of all elements of the model. In a word, what has been missing is in this line of inquiry is “learning” or more significantly “deep and meaningful learning” which is the assumed goal of all higher education. Additionally previous research (Garrison, Anderson, & Archer, 2000; Garrison, Anderson, & Archer, 2001; Kanuka, Rourke, & Laflamme, 2007; Rourke & Kanuka, 2009; Stein, Wanstreet, Engle, et. al, 2006; Vaughn & Garrison, 2006) suggests that students in online courses tend not to reach higher stages of cognitive presence, i.e. integration, application and resolution. It is therefore critical that we attempt to address these weaknesses and gaps in the research. In this paper we therefore investigated hypotheses generated by recent theory driven research with regards to the development of knowledge construction within the CoI framework (Shea and Biderjano, 2009; Garrison and Cleveland-Innis, in press). Previous research, based on factor analysis and structural equation modeling, suggests that multivariate measures of cognitive presence presumed to reflect significant learning, especially at the higher stages of integration, application and resolution, develops as a result of instructor teaching presence and is mediated by social presence. In a study of more than 2000 online learners, Shea and Bidjerano (2009) found that more than 70% of variance in students’ reported cognitive presence could be predicted based on perceived teaching and social presence (p. 548). Garrison and Cleveland-Innes replicated these findings in a study of 205 of online learners (in press). In both studies social presence developed as a function of instructor teaching presence and was found to mediate the development of cognitive presence. Additionally, through the use of CHAID analysis Shea and Bidjerano (2009) concluded that students who reported high levels of cognitive presence could be distinguished most significantly from those who reported low levels by their answers to an item reflecting social presence. Given these findings social presence appears to play an important role in the advancement of cognitive presence, a measure of significant learning within the CoI framework. In the current paper we present preliminary results to further investigate these relationships. Central to this line of inquiry are efforts to overcome methodological weaknesses in previous research. For example in the studies by Shea & Bidjerano (2009) and Garrison & Cleveland Innes (in press) as well as others (Arbaugh et al, 2008) teaching and social presence were measured through survey items. While student reports serve an important function in this line of research we feel it is important to measure constructs within the CoI framework more directly as well. Quantitative content analysis is useful in this regard as it allows for more direct measurement. Also missing from past research is an investigation of instructor social presence and its relationship to the development of students’ reports of social presence. In the current study therefore we use quantitative content analysis to question whether student social presence develops as a result of instructor teaching presence or instructor social presence. We do this by examining two online courses that vary with regard to instructor teaching and social presence and compare the levels and patterns of student social presence in each. Further we document patterns of teaching and cognitive presence. Research Questions: 1) What methodological issues arise in coding online courses for teaching, social, and cognitive presence using quantitative content analysis? What aspects of these constructs are problematic? 2) Using quantitative content analysis what are the patterns of teaching, social, and cognitive presence that emerge through joint codings of course discussion transcripts? What kinds and levels of presence do instructors and students exhibit? 3) What are the relationships between teaching, social, and cognitive presence revealed through content analysis of two courses? Do students’ indicators of presence track instructor presence? 4) How does low instructor presence correlate with students’ social and cognitive presence relative to higher instructor presence? Do students demonstrate higher levels of cognitive presence when instructors are more active as measured through instructor teaching presence? 5) How do social network analysis metrics compare to quantitative content analysis in measuring teaching presence? Methods We argue that previous research on the CoI model has been limited in several respects. First most empirical research has focused on a single component of the framework, either teaching, social, or cognitive presence (e.g Anderson et al, 2000; Shea et al, 2003, 2006; Swan & Shih, 2001; Rourke et al 1999), though exceptions exist (e.g. Akyol and Garrison, 2008; Shea & Bidjerano, 2009). We believe it is time to simultaneously examine and attempt to validate all components and the model as a whole, a strategy we employ here. Second, research on the CoI has been limited largely to survey methods or content analysis, the latter focusing solely on the discussion transcripts within a single course (see Arbaugh and Hwang, 2006; Shea et al 2006 for examples of the former, Anderson et al, 2001 Swan & Shih, 2001; Rourke et al 1999 for examples of the latter). We believe that this research can be improved in a number of ways. First we employ mixed methods and present a study using both social network analysis and quantitative content analysis to illuminate differences in discourse patterns. A third strategy for improvement is to examine more than one course in which levels of teaching and social presence vary. In the present study we therefore analyzed two courses that use the same instructional design template, but have different approaches to the facilitation of discourse and direct instruction and in which the instructors’ social presence levels were different. The data for this research includes all of the content of two fully online courses offered by a State college in the northeast that specializes in distance and adult education for non-traditional learners. The two courses, in Business Management, were designed by content experts and multimedia instructional designers and taught by instructors who are not the course designers. The two courses consist of five modules of instruction and contain a variety of learning activities including discussions, and individual and group assignments. As mentioned above however the design of the courses was based on a template and the two were identical in this regard. To code for social presence we used the indicators originally devised by Rourke et. al. (1999) with revisions by Swan and Shih (2001). We made some minor modifications to the social presence indicators. For teaching presence we used the codes devised by Anderson, Garrison, Rourke and Archer (2001) and made significant revisions in the area of direct instruction, which previous research has indicated failed to cohere as an interpretable factor (Shea et al, 2006). Cognitive presence codes were taken from Garrison, Anderson and Archer (2001) and were modified by removing “divergent” and “convergent” from the Exploration and Integration category descriptions and by deleting the Brainstorming indicator from Exploration because it could not be differentiated from Information Exchange. The coding schemes developed through this research are included in appendix B. Content analysis was conducted on a total of 10 discussions in all five modules in each course. There were a total of 490 posts in Course A and 454 posts in Course B. At this point in the research at least two independent coders have evaluated each sentence in all course discussions for specific social, teaching, and cognitive presence indicators, with the unit of analysis at the message level. At present researchers are also continuing work on the development of coding schemes for analyzing course assignments outside of threaded discussions to evaluate cognitive presence levels. Previous research using content analysis suggests that considerable care is needed to reach valid results and that achieving high levels of agreement between coders is critical to this goal. Earlier research has been somewhat remiss in this regard with authors frequently supplying inadequate information, or no indicators of inter-rater reliability measures. Previous work has also depended on very small samples of jointly coded and compared content assuming that subsequent coding proceeded without significant disagreement. We took several steps to improve upon these weaknesses. First we calculated two measures of inter-rater reliability, and second we did not depend on sampling, but rather coders compared results of coding for all discussions in all of the modules of both courses to ensure adequate levels of validity and reliability. Inter-rater reliability was initially computed using Cohen’s Kappa. After calculating initial inter-rater reliability, the coders met to negotiate disagreements. Previous research suggests that symmetrical imbalances in the marginal distributions of the coding table is problematic and can lead to high levels of observed agreement yet a very low kappa (Feinstein & Cicchetti, 1990). Because of this, Holsti’s Coefficient of Reliability, which measures percent agreement was also used to calculate inter-rater reliability.